What is big data?

The definition of big data is data that contains greater variety, arriving in increasing volumes and with more velocity. This is also known as the three "Vs".

Put simply, big data is larger, more complex data sets, specially from new data sources.

The Three Vs of Big Data

Volume

The amount of data matters. With big data, you'll have to process high volumes of low-density, unstructured data. THis can be data of unknown values, such as Twitter, click streams on a web page or a mobile app, or sensor-enabled equipment. For some organizations, this might be tens of terabytes of data. For others, it might be hundreds of petabytes.

Velocity

Velocity is the fast rate at which data is received and (perhaps) acted on. Normally the highest velocity of data streams directly into memory versus being written on disk. Some internet-enabled smart products operate in real time or near real time and will require real-time evaluation and action.

Variety

Variety refers to the many types of data that are available. Traditional data types were structured and fit neatly in a relational database. With the rise of big data, data comes in new unstructured data types. Unstructured and semi-structured data types, such as text, audio and video, require additional preprocessing to derive meaning and support metadata.

The Value -- and Truth -- of Big Data

Two more Vs have emerged over the past few years: value and veracity.

Finding value in big data is not only about analyzing it (which is a whole other benefit). It is an entire discovery process that requires insightful analysts, business users, and executives who ask the right questions, recognize patterns, make informed assumptions, and predict behavior.

The History of Big Data

Around 2005, people began to realize just how much data users generated through Facebook, YouTube, and other online services. Hadoop (an open source framework created specifically to store and analyze big data sets) was developed by that same year. NoSQL was also began to gain popularity at this time.

The development of open source frameworks such as Hadoop was essential for the growth of big data because they make big data easier to work with and cheaper to store. In the years since then, the volume of big data has skyrocketed. Users are still generating huge amounts of data - but it's not just humans who are doing it.

With the advent of Internet of Things (IoT) more objects and devices are connected to the internet, gathering data on customer usage patterns and product performance. The emergence of machine learning has produced still more data.

While big data has come far, its usefulness is only beginning. Cloud computing has expanded big data possibilities even further.

Big data benefits

• Big data makes it possible for you to gain more complete answers because you have more information
• More complete answers mean more confidence in the data -- which means a completely different approach to tackling problems

Big Data Use Cases

Product Development

Companies like Netflix and Procter & Gamble use big data to anticipate customer demand. They build predictive models for new products and services by classifying key attributes of past and current products or services and modeling the relationships between them.

Predictive Maintenance

Factors that can predict mechanical failures may be deeply buried in structured data, such as teh year make, and model of equipment, as well in unstructured data that covers millions of log entries, sensor data, error messages, and engine temperature. By analyzing these indications of potential issues before the problems happen, organizations can predict the time and cost of repairs.

Customer Experience

The race for customers is on. A clearer view of customer experience is more possible now that even before. Big data enables you to gather data from social media, web visits, call logs, and other sources to improve the interaction experience and maximize the value delivered. Start delivering personalized offers, reduce customer churn, and handle issues proactively.

Fraud and Compliance

When it comes to security, it's not just a few rogue hackers - you're up against entire expert teams. Security landscapes and compliance requirements are constantly evolving. Big data helps you identify patterns on data that indicates fraud and aggregate large volumes of information to make regulatory reporting much faster.

Machine Learning

Machine learning is a hot topic right now. And big data is one of the reasons why. We are now able to teach machines instead of program them. The availability of big data to train machine learning models makes that possible.

Operational Efficiency

Operational efficiency may not always make the news, but it's an area in which big data is having the most impact. With big data you can analyze and asses production, customer feedback and returns, and other factors to reduce outages and anticipate future demands.

Drive Innovation

Big data can help you innovate by studying interdependencies among humans, institutions, entries, and process and then determining nwe ways to use those insights. Use data insights to improve decisions about financial and planing considerations. Examine trends and what customers want to deliver nwe products and services. Implement dynamic pricing. There are endless options.